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SEMIAUTOMATIC KEYSTONE CORRECTION SYSTEM AND METHOD 

This application claims priority from U.S. provisional patent application serial 
number 60/443,422 filed January 28, 2003, titled Method and Apparatus for Keystone 
Correction, which we incorporate in its entirety. 

FIELD OF THE INVENTION 

This invention relates to a system and method capable of projecting images and, more 
particularly, to a system and method capable of semiautomatically correcting keystone 
distortion of projected images. 

BACKGROUND OF THE INVENTION 

Projection systems are widely used in training, sales, and business environments. 
Referring to Figure 1, a projection system 100 includes a projector 102 positioned on a 
horizontal surface 104. The surface 104 is typically a desk or tabletop. An elevator 120 
protrudes from the bottom sides of the projector 102 creating an angle 110 between the 
surface 104 and the projector 102. Only one elevator 120 is visible in Figure 1 although a 
person of reasonable skill in the art should understand that a plurality of elevators 120 might 
be employed in the system 100. The angle 110 varies depending on the position of the 
elevator 120. The elevator 120 tilts the projector 102's position relative to the surface 104 
such that projected image 1 18 moves up or down on a projection surface 1 14, increasing or 
decreasing the angle 110. The projection surface 114 might be a wall, screen, or any other 
surface capable of displaying a projected image 118. 

The projector 102 manipulates image signals 108 it receives from a personal computer 
106. A person of reasonable skill in the art should recognize that the projector 102 might 
receive different types of image signals, e.g., digital or analog signals, from the personal 
computer 106. The image signals 108 represent still, partial, or full motion images of the 
type rendered by the personal computer 106. 

The projector 102 casts the image signals 108 onto the projection surface 1 14. The 
resulting projected image 118 centers about a projection axis 116. An angle 1 12 exists 
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between the projection axis 1 16 and the projection surface 114. The angle 112 changes 
responsive to changes in the angle 110. 

The projected image 1 18 is substantially undistorted if the projection axis 1 16 is 
perpendicular to the projection surface 1 14. That is, the image 1 18 is undistorted if the angle 
5 1 12 is 90 degrees. The projected image 118, however, distorts if the projection axis 1 16 is 
not perpendicular to the projection surface 1 14. This distortion is termed keystone distortion 
(or keystoning) because the image will appear wider at the top than at the bottom as shown in 
the jagged lined image 122. 

There are well known ways of correcting keystone distortion. The elevator 120 
10 might, for example, be coupled to optics encased in the projector 102 that compensate for 
keystone distortion. The optics, however, are costly and prone to dust collecting that results 
in obscured projected images. 

Signal processing circuits are often used, for example, to oppositely distort the image 
to thereby compensate for keystone distortion prior to projecting the image. But these 
15 circuits require floating point processes that exceed real time system capability. 

The projector 102 might include gauges 124 used to manually adjust the projected 
image 1 18 to eliminate or minimize keystone distortion. The manual adjustments tend to 
move the projected image 118 out of the projection surface 1 14. And the manual 
adjustments, unfortunately, are time consuming, cumbersome, and generally an unwelcome 
20 set up complication. 

Accordingly, a need remains for a semiautomatic keystone correction system and 
method. 

BRIEF DESCRIPTION OF THE DRAWINGS 

25 The foregoing and other objects, features, and advantages of the invention(s) will 

become more readily apparent from the detailed description of invention embodiments that 
references the following drawings. 

Figure 1 is a diagram of a projection system. 

Figure 2 is a block diagram of an embodiment of a projection system. 
30 Figure 3 is a block diagram of an embodiment of a controller. 

Figure 4 is a block diagram of an embodiment of a keystone controller. 
Figures 5 A-D are graphic representations of the operation of the system 200 from the 
user's perspective. 
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Figures 6A-C are diagrams of a coordinate system used to describe keystone 
distortion. 

Figures 7 and 8 are graphical representations of the operation of the keystone 
controller shown in Figures 3 and 4. 
5 Figures 9 and 10 are graphical representations of comparisons between the inventive 

semiautomatic keystone correction system and method and more traditional keystone 
correction systems and methods. 

Figure 11 is an embodiment of a lookup table. 

10 DESCRIPTION OF THE IP^VEOTION 

Figure 2 is a block diagram of a projection system 200 according to an embodiment of 
the present invention. Referring to Figures 1 and 2, the system 200 is capable of projecting 
an image 1 1 8 on a projection surface 114. The system includes a receiver 220 for receiving 
an analog image data signal 210, e.g., an RGB signal, from a source 202. The receiver 220 

15 might be an analog-to-digital converter (ADC) or the like. The source 202 might be a 

personal computer or the like. The receiver 220 converts the analog image data signal 210 
into digital image data 230 and provides it to the panel controller 2S0. 

Likewise, a video receiver or decoder 222 decodes an analog video signal 212 from a 
video source 204. The video source 204 might be a video camcorder and the like. The 

20 decoder 222 converts the analog video signal 212 into digital image data 230 and provides it 
to the panel controller 250. 

A modem or network interface card (NIC) 224 receives digital data 214 from a global 
computer network 206 such as the Internet®. The modem 224 provides digital image data 
230 to the panel controller 250. 

25 A Digital Visual Interface (DVI) receiver 226 receives digital RGB signals 216 from 

a digital RGB source 208. The DVI receiver 226 provides digital image data 230 to the panel 
controller 250. 

A person of reasonable skill in the art should recognize other sources and other 
converters come within the scope of the present invention. 
30 The panel controller 250 generates panel data 232 by manipulating the digital image 

data 230. The panel controller 250 provides the panel data 232 to a flat panel device 260. 
The panel 260 is any device capable of projecting the digital image data 230. In an 
embodiment, the panel 260 includes a pixelated display that has a fixed pixel structure 
together with the optics and electronics necessary to project the digital image data 232 on a 
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surface 114 (Figure 1). Examples of pixelated displays are active and passive LCD displays, 
plasma displays (PDP), field emissive displays (FED), electro-luminescent (EL) displays, 
micro-mirror technology displays, low temperature polysilicon (LTPS) displays, and the like 
for use in television, monitor, projector, hand held, and other like applications. The optics 
5 and electronics necessary to project the panel data 232 are well known to those of reasonable 
skill in the art. 

In an embodiment, the panel controller 250 might scale the digital image data 230 for 
proper projection by the panel 260 using a variety of techniques including pixel replication, 
spatial and temporal interpolation, digital signal filtering and processing, and the like. In 

10 another embodiment, the controller 250 might additionally change the resolution of the 

digital image data 230, changing the frame rate and/or pixel rate encoded in the digital image 
data 230. Scaling, resolution, frame, and/or pixel rate conversion, and/or color manipulation 
are not central to this invention and are not discussed in further detail. A person of 
reasonable skill in the art should recognize that the controller 250 manipulates the digital 

15 image data 230 and provides panel data 232 to a panel 260 that is capable of properly 
projecting a high quality image regardless of display type. 

Read-only (ROM) and random access (RAM) memories 240 and 242, respectively, 
are coupled to the display system controller 250 and store bitmaps, FIR filter coefficients, and 
the like. A person of reasonable skill in the art should recognize that the ROM and RAM 

20 memories 240 and 242, respectively, might be of any type or size depending on the 

application, cost, and other system constraints. A person of reasonable skill in the art should 
recognize that the ROM and RAM memories 240 and 242 might not be included in the 
system 200. A person of reasonable skill in the art should recognize that the ROM and RAM 
memories 240 and 242 might be external or internal to the controller 250. Clock 244 controls 

25 timing associated with various operations of the controller 250. A person of reasonable skill 
in the art should recognize that the projector 102 might house all or part of the controller 250, 
clock 244, RAM 242, ROM 240, panel 260, as well as the optics and electronics necessary to 
project the panel data 232. 

Figure 3 is a block diagram of an embodiment of the controller 250 shown in Figure 

30 2. Referring to Figures 1-3, the controller 250 includes an interface 352 coupled to a driver 
354 and a keystone controller 356. The interface 352 interacts with a user 358 in a variety of 
manners. The interface 352 shows the user 358 instructions on e.g., on the projection surface 
1 14, and accepts input 251 from the user 358 using a variety of input means, including 
infrared input devices, keypads, and/or mice (not shown). The interface 352 might be a 



Patent Application 



Page 4 OF 17 



Do. No. 7293-56 



graphical user interface (GUI) that takes advantage of the controller 250' s graphics 
capabilities to ease the interaction between it and the user 358. The interface 352 might 
support such features as infrared input devices, keypads, mice, pointers, other pointing 
devices (e.g., trackball), graphically presented commands (e.g., icons), desktops, windows, 
and/or menus. 

The user 358 indicates to the interface 352 properties of the projected image 1 18 
and/or the projection surface 114. In an embodiment, the interface 352 generates an 
electronic icon or other GUI that the user 358 manipulates to align a center of the projected 
image 118 with a center of the projection surface 1 14. The user 358 actuates the GUI to 
inform the interface 352 of the centers' alignment (and its position). It should be apparent to 
a person of skill in the art that the user 358 can align the center of the projected image 118 
with surfaces or areas other than the projection surface 114. 

The user 358 uses the interface 352's GUI (e.g., on screen display) to select a plurality 
of comers of the desired (undistorted) image within the projected image 118. The plurality of 
comers might be, e.g., two, three, or four comers. In an embodiment, the user's selected 
comers lie within the projected image 118. 

The interface 352 transmits the user's input 353, e.g., the center and plurality of 
comers, to the driver 354. The driver 354, in turn, interprets the user's input 353 to calculate 
the horizontal and vertical rotation angles Ph and Pv- 

The keystone controller 356 uses the information provided by the driver 354, e.g., the 
rotation angle, to (pre) distort the image 118 before its (re) projection by the projector 102 on 
the surface 114. By doing so, the controller 250 (pre) distorts the image prior to and to 
account for keystone distortion visible on the projected image 118. The (pre) distorted image 
will exhibit little to no keystone distortion once projected on the surface 1 14. 

Figures 5A-D are graphic representations of the operation of the system 200 from the 
user's perspective. Referring to Figures 1-3, and 5A-5D, the projected image 518 is 
projected on a surface 514. The projected image 518 has a center 562. The projection 
surface 514 has a center 564. The user aligns the center 562 with the center 564 (Figure 5B) 
using the interface 352. 

The user selects a plurality of comers 566 indicating where it wants the comers of the 
undistorted (re) projected image 568. The plurality of comers 566 might lay within the 
projected image 518 and the screen 5 14. The controller 250 scales or (pre) distorts the 
original image such that the newly projected image 568 does not exhibit keystone distortion. 
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The projected image 568 has a center aligned with the center 564 and comers coincident with 
comers 566 selected by the user. 

Figure 4 is a block diagram of the keystone controller 356. Referring to Figures 1-4, a 
keystone controller 356 receives digital image data 355 from the driver 354. Digital image 
5 data 355 includes digital image data 251 provided by e.g., the computer 357. Digital image 
data 355 includes the user's input as provided to the interface 352, e.g., requesting keystone 
correction and identifying a center and comers as we discussed above. 

If the angle 1 12 is 90 degrees, the projector 102 will project the digital image 355 as 
an undistorted image 118. That is, if the projection axis 1 16 is perpendicular to the surface 

10 1 14, the projector 102 will project the digital image 355 as an undistorted image 118. If the 
angle 1 12 is not 90 degrees (that is, if the projection axis 1 16 is not perpendicular to the 
surface 1 14), the digital image 355 will project a distorted image 122. The controller 250 
and, more particularly, the keystone controller 356, (pre) distorts the digital image data 355 
prior to its (re) projection such that when the projector 102 (re) projects the (pre) distorted 

15 image data 232 on the surface 1 14, the image appears as if the angle 1 12 where 90 degrees. 
Put differently, the keystone controller 356 outputs digital image data 232 that is a distorted 
version of the digital image data 355. The keystone controller 356 maps (scales or distorts) 
the original image 5 18 to the desired image 568. The keystone controller 356 effectuates the 
distortion by vertically and horizontally scaling the digital image data 355 responsive to the 

20 user's input, e.g., the user's center and comer selections. 

The keystone controller 356 includes a vertical scalar 490 for vertically scaling the 
digital image data 355. And the keystone controller 356 includes a horizontal scalar 494 for 
horizontally scaling the image data 253. 

A vertical enable register 472 enables the vertical scalar 490. The vertical scalar 490 

25 generates the vertically scaled data 253 responsive to vertical offset registers 474 including 
the registers YE (column even), YO (column odd), and YI (column increment (1^* order 
term)) where the image data 355 is an interlaced signal. And the vertical scalar 490 generates 
the vertically scaled data 253 responsive to the vertical registers 492 including registers VDN 
(line position), VDNI (increment per line (1^^ order term)), and VDNPI (increment per 

30 column). The vertical scalar 490 moves up one row or line responsive to the UP register 484. 
The microprocessor 488 scales the image date 355 responsive to the registers 472, 474, 484, 
and 492 and to the horizontal and vertical synchronization signals 455. 

Likewise, a horizontal enable register 478 enables the horizontal scalar 494. The 
horizontal scalar 494 generates the horizontally and vertically scaled data 232 responsive to 
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horizontal offset registers 480 including the registers XE (row even), XO (row odd), and XI 
(row increment (1^^ order term)) where the image data 355 is an interlaced signal. And the 
horizontal scalar 494 generates the horizontally and vertically scaled data 232 responsive to 
the horizontal registers 482 including registers HDN (row position), HDNI (increment per 
line (1^^ order term)), HDNII (increment increment per line (2"^ order term)), and HDNPI 
(increment per column). The horizontal scalar moves one column or pixel responsive to the 
UP register 486. The microprocessor 488 scales the image data 355 responsive to the 
registers 478, 480, 482, and 486 and to the horizontal and vertical synchronization signals 
455. 

Before turning to the operation of the keystone controller 356, we examine some 
keystone basics. Figures 6A-C are diagrams of a coordinate system used to describe 
keystone distortion. Referring to Figures 6A-C, a projected image falls on a plane 600 
defined by coordinates (x, y, 0). PP defines a projection point at coordinates (0,0,d), where d 
is a distance from the image plane 600 to the projection point PP. NPL defines a normal 
projection line to the plane 600. Figures 6B and 6C are side and top views are the coordinate 
system shown in Figure 6A where db is the difference between a horizontal projection axis 
and the normal projection line NPL. Ovt and Ovb are the vertical top and bottom angles, 
respectively. Ovl and Ovr are the left and right angles, respectively. HnO is the vertical 
height and WnO is the horizontal width of the image in a nominal position. 

Figures 7 and 8 are graphical representations of the operation of the keystone 
controller 356. Referring to Figure 7, the image data 355 (or 25 1) is represented by (Xn, yn). 
The output data 232 is represented by (xk, yk). Formula 1 gives the function f\. 

Formula 1 

(xk, yk) = fi (xn, ym Ph, and pv) 
where Ph is the horizontal rotation angle and Pv is the vertical rotation angle of the 
desired keystone corrected (or distorted) image 232. 

The keystone controller 356 calculates fi as follows. 
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Cos[fih] XX- Sin[ph\ x Sin[Pv] x y 
xp\.x,y\ - ^ ^ sin[ph\ X jc + Co5[>g/i] x Sin[Pv] x 

Cos[M^y-{db-^) 

= ^ ^ 5/>i[>g/?3xjc + Co5[>g/?]x5/>i[/?v;]x3; 



To calculate the registers used to distort the image, the controller 256 fixes the center 
of the image with the user's selected center and uses the user's comer selection as follows. 
The comers associated with the keystone distorted image and comers associated with the 
nominal image are related by a function fs. 

(Xkc, ykc) = fs (Xnc, ync, Ph, Pv) 

Where the function fs is given by the following formulas. 
TopLeft 

^ HnO 

, —shxsv-chxWnO , „ HnO 



chxsv- shx WnO 2 , chxsv — shx WnO 

1 + 1 + 

d d 

TopRight 

^ HnO 

- shx sv + chx WnO „ HnO ? 

xtr = ; — „- ^ , ytr - do 1 ; . --- ^ 

. chxsv + shxWnO 2 , chxsv + shxWnO 

1 + 1+ 

d d 

BottomLeft 

cv db I 

, , —shxsv — chx WnO , , „ HnO o 

Xbl = J J ^T^^ybl = db + ; y rrr 

-chxsv + shx WnO 2 , — chxsv + shx WnO 

1 + 1 + 

d d 

BottomRight 

_ „ HnO 

shxsv^chxWnO ^ HnO ^^'^ y 

xbr = ; ; — „- ^ , ybr = db + z , „ . ^ 

^ -chxsv-hshxWnO 2 , -chxsv-^-shxWnO 
1+ 1 + 



Where : 
sh Sm[/}h] 
ch Cos[^] 
sv — > Sinlfiv] 
cv -> Cos[fiv] 
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Given a set of comers associated with the keystone distorted image (xkc, ykc) and 
having fixed the center of the distorted image at the user's selected center, the sine function 
of the horizontal rotation angle Ph is calculated using simple arithmetic operations. Likewise, 
given (Xkc, Ykc) and the cosine function of the horizontal rotation angle Ph, the sine function of 
the vertical rotation angle Pv is calculated using simple arithmetic operations. Solving for fs 
results in the following. 

r,, r ^, -. d X (—xbl X xtl + xbr x xtr) 
Sinyph] = 



WnO X (xbr - xtl) x (xbl - xtr) 

d X {xbl X xbr — xd x xtr) 

SinlM = ^ ^ ^^^^ ~ ^^^^ ^ ^^^^ 
ch 

The keystone controller 356 calculates the required register values for predistorting or 
keystone correcting the image. In an embodiment, the keystone controller 356 accesses a 
precalculated look up table comprising all of the register values for several rotation stages. 
Doing so speeds up execution time by allowing register value precalculation instead of 
having to calculate the register values on the fly. 

Figure 11 is an embodiment of a lookup table 1 100. The table includes a 
predetermined number of combinations, e.g., 0, +/-10, and +/-20 on the x-axis and 0, +/-20, 
and +/-40 on the y-axis. 

If a rotation angle combination falls between two points, the system interpolates 
between the values given by the table. The image rotation angles are sent from the driver 354 
to the keystone controller 356 to initiate calculation of the scalar register values. The 
following is an exemplary bilinear interpolation formula. 

wx = ^ ^ 

estimate = (1 - wx) x (1 - wy) x Table[p, q\-\-wxx{\- wy) x Table[p + 1, 9] + 
(1 - wx) xwyx Table[p, q-\-l]-\-wxxwyx Table[p + 1, ^ + 1] 

Where the table provides the coordinates (p, q) of the nearest point and the keystone 



Patent Application 



Page 9 OF 17 



Do. No. 7293-56 



controller 356 interpolates the nearest point to the vertical and horizontal rotation angles of 
the interpolated point. 

A person of reasonable skill in the art should identify several different interpolation 
methodologies as coming within the scope of the present invention. A person of reasonable 
5 skill in the art should realize the table could be implemented in a variety of manners using a 
variety of hardware, including implementing it as part of semiconductor memory 240 and/or 
242 or memory embedded within the controller 250. 

Formula fi is a complicated formula containing trigonometric functions that are 
computation intensive. The present invention constructs simple polynomials fz and fs that 
10 approximate fi. Referring to Figures 1-8, the image data 251 is vertically scaled by vertical 
scalar 490 responsive to an f2 function to produce the vertically scaled data 253. Formula 2 
gives the function f2. 

Formula 2 

fi (Xn, yn, vdn, vdni, vdnpi, yo, yi) 
15 The vertical scalar 490 operates responsive to the register VDN and the x iand y 

position (line and column number) of the projected image 518. 

The vertical scalar 490 uses a vertical scale factor of VDN(x,y) = VDN + f(x,y). 
The vertical offset registers YE and YO control the top edge of the image. The top of 
the image is a function of YEA^O and X. The top of the image is also a weak function of the 
20 registers VDN, VDNI, and VDNPI. 

The vertical offset register 486 is related to YE (or YO) + f(x). 
The vertical scalar 490 operates under the control of the microprocessor 488. 
The vertically scaled data 253 is horizontally scaled by the horizontal scalar 494 
responsive to an fa function to produce the horizontally scaled data 232. Formula 3 gives the 
25 function fa. 

Formula 3 

fs (xn, yic, hdn, hdni, hdnii, hdnpi, hdnpii, xo, xi, xii) 
The horizontal scalar 494 operates responsive to the register HDN and the x and y 
position (line and column numbers) of the projected image 518. 
30 The horizontal scalar 494 uses a horizontal scale factor of HDN(x,y) = HDN + f(x,y). 

The horizontal offset register 486 directly controls the left edge of the image. 
The horizontal scalar 494 operates under the control of the microprocessor 488. 
Calculate f2 and fs as follows. 
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The combination of f2 and fs approximates fi. 

The vertical scalar 490 calculates its registers as follows. 

With fi and a set of (xn, yn), calculate (xk, yk). 

Given the (Xn, yn) input, the (Xn, yk) output, and f2, the keystone controller 356 
calculates vdn, vdni, vdnpi, yo, and yi. 

The horizontal scalar 494 calculates its registers as follows. 

Given the (xk, yk) output, the (Xn, yk) input, and fa, the keystone controller 356 
calculates hdn, hdni, hdnii, xo, xi, and xii. 

Figures 9 and 10 are graphical representations of comparisons between the inventive 
semiautomatic keystone correction system and method and more traditional keystone 
correction systems and methods. Referring to Figure 9, image 900 represents the nominal 
image with no keystone correction. Image 902 is predistorted using center fix, first 
vertical/last horizontal (FVLH) inventive keystone correction system and method. And 
image 904 is predistorted using traditional keystone correction (e.g., minimal distortion 
method). The images 902 and 904 are keystone corrected 45 degrees horizontally and 20 
degrees vertically. As shown, the image 904 moves around the plane because the minimal 
distortion method seeks to position the predistorted image in the area of the original image 
where there is least or minimal distortion. Doing so, however, results in a much smaller 
image relative to the image 902 corrected using the inventive keystone corrective system and 
method. In contrast, the image 904 remains fixed and aligned to the center of the image 900. 

Referring to Figure 10, image 1000 represents the nominal image with no keystone 
correction. Image 1002 is predistorted using center fix, first vertical/last horizontal (FVLH) 
inventive keystone correction system and method. And image 1004 is predistorted using 
traditional keystone correction. The images 1002 and 1004 are keystone corrected -45 
degrees horizontally and 20 degrees vertically. As shown, the image 1004 is much smaller 
relative to the image 1002 corrected using the inventive keystone corrective system and 
method. And the image 1004 moves around the plane where the image 1002 remains fixed 
and aligned to the center of the image 1000. 
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Having illustrated and described the principles of our invention(s), it should be readily 
apparent to those skilled in the art that the invention(s) can be modified in arrangement and 
detail without departing from such principles. We claim all modifications coming within the 
spirit and scope of the accompanying claims. 
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